Experimental results

The following code summarizes and plots the results out our experiments

Type any R code in the chunk, for example:

── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ forcats   1.0.0     ✔ purrr     1.0.2
✔ ggplot2   3.5.1     ✔ stringr   1.5.1
✔ lubridate 1.9.3     ✔ tibble    3.2.1
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag()    masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors

Warning: Removed 50 rows containing missing values or values outside the scale range
(`geom_line()`).

Warning in max(ids, na.rm = TRUE): no non-missing arguments to max; returning
-Inf

Warning: There were 4 warnings in `reframe()`.
The first warning was:
ℹ In argument: `auc_low = (mean(auc) - qt(0.975, df = n() - 1) *
  sd(auc)/sqrt(n()))`.
Caused by warning in `qt()`:
! NaNs produced
ℹ Run `dplyr::last_dplyr_warnings()` to see the 3 remaining warnings.

Warning: There were 4 warnings in `reframe()`.
The first warning was:
ℹ In argument: `auc_low = (mean(auc) - qt(0.975, df = n() - 1) *
  sd(auc)/sqrt(n()))`.
Caused by warning in `qt()`:
! NaNs produced
ℹ Run `dplyr::last_dplyr_warnings()` to see the 3 remaining warnings.

Warning: There were 4 warnings in `reframe()`.
The first warning was:
ℹ In argument: `auc_low = (mean(auc) - qt(0.975, df = n() - 1) *
  sd(auc)/sqrt(n()))`.
Caused by warning in `qt()`:
! NaNs produced
ℹ Run `dplyr::last_dplyr_warnings()` to see the 3 remaining warnings.

Warning: There were 4 warnings in `reframe()`.
The first warning was:
ℹ In argument: `auc_low = (mean(auc) - qt(0.975, df = n() - 1) *
  sd(auc)/sqrt(n()))`.
Caused by warning in `qt()`:
! NaNs produced
ℹ Run `dplyr::last_dplyr_warnings()` to see the 3 remaining warnings.

Warning: There were 4 warnings in `reframe()`.
The first warning was:
ℹ In argument: `auc_low = (mean(auc) - qt(0.975, df = n() - 1) *
  sd(auc)/sqrt(n()))`.
Caused by warning in `qt()`:
! NaNs produced
ℹ Run `dplyr::last_dplyr_warnings()` to see the 3 remaining warnings.

Warning: There were 4 warnings in `reframe()`.
The first warning was:
ℹ In argument: `auc_low = (mean(auc) - qt(0.975, df = n() - 1) *
  sd(auc)/sqrt(n()))`.
Caused by warning in `qt()`:
! NaNs produced
ℹ Run `dplyr::last_dplyr_warnings()` to see the 3 remaining warnings.

Warning: Removed 14 rows containing missing values or values outside the scale range
(`geom_line()`).

Warning: There were 4 warnings in `reframe()`.
The first warning was:
ℹ In argument: `auc_low = (mean(auc) - qt(0.975, df = n() - 1) *
  sd(auc)/sqrt(n()))`.
Caused by warning in `qt()`:
! NaNs produced
ℹ Run `dplyr::last_dplyr_warnings()` to see the 3 remaining warnings.

Warning: There were 4 warnings in `reframe()`.
The first warning was:
ℹ In argument: `auc_low = (mean(auc) - qt(0.975, df = n() - 1) *
  sd(auc)/sqrt(n()))`.
Caused by warning in `qt()`:
! NaNs produced
ℹ Run `dplyr::last_dplyr_warnings()` to see the 3 remaining warnings.

Warning: There were 4 warnings in `reframe()`.
The first warning was:
ℹ In argument: `auc_low = (mean(auc) - qt(0.975, df = n() - 1) *
  sd(auc)/sqrt(n()))`.
Caused by warning in `qt()`:
! NaNs produced
ℹ Run `dplyr::last_dplyr_warnings()` to see the 3 remaining warnings.

Warning: There were 4 warnings in `reframe()`.
The first warning was:
ℹ In argument: `auc_low = (mean(auc) - qt(0.975, df = n() - 1) *
  sd(auc)/sqrt(n()))`.
Caused by warning in `qt()`:
! NaNs produced
ℹ Run `dplyr::last_dplyr_warnings()` to see the 3 remaining warnings.

Table summarizing the experiment results

Joining with `by = join_by(dataset, estimator, height)`

Summary Plot

data <- read.csv("results_aggregated/SUSY_SVM_break_2024-06-14_09:31:52.csv")
break_down_estimates <- data %>% group_by(outliers) %>% reframe(break_prob = mean(auc<0.5))
plot(break_down_estimates)

break_down_estimates